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Abstract 

This  paper  presents  a  new  method  for  minimizing  a  positive  definite  quadratic 
function  subject  to  linear  inequality  constraints.  It  is  based  on  a  continuous  defor¬ 
mation  of  the  quadratic  starting  from  one  giving  rise  to  an  almost  trivial  problem 
and  ending  with  the  desired  quadratic. 

The  method  is  particularly  suitable  for  problems  involving  large  numbers  of 
inequality  constraints,  since  the  size  of  the  variable  space  is  independent  of  the 
number  of  inequalities. 

The  method  is  readily  extended  to  linear  programming  by  embedding  the  linear 
objective  function  in  a  second  degree  polynomial  with  positive  definite  quadratic 
component,  solving  the  resulting  problem  by  the  above  method,  then  driving  the 
quadratic  component  to  zero;  that  is,  deforming  the  second  degree  programming 
problem  back  to  the  original  linear  one. 
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A  Deformation  Method  for  the  Minimization 
of  a  Quadratic  Function  Subject 
to  Linear  Inequality  Constraints 


1.  INTRODUCTION 

In  recent  years  there  has  been  considerable  research  devoted  to  the  problem  of 
minimizing  a  positive  definite  quadratic  function  of  n  variables  over  a  polyhedral 
region  in  Euclidean  n  space  and  a  variety  of  methods  have  appeared  in  the  litera¬ 
ture.  This  paper  presents  a  method  closely  related  to  the  capacity  methods  of 
1  2 

Houthakker  and  Markowitz.  My  reason  for  presenting  another  method,  and  per¬ 
haps  adding  to  the  confusion,  is  that  their  methods  are  tailored  to  a  particular  ap¬ 
plication  and  are  less  well  suited  to  other  applications.  Furthermore,  the  present 
method  is  readily  extended  to  linear  programming,  whereas  the  capacity  methods 
do  not  seem  capable  of  this  extension. 

This  method,  as  well  as  those  of  Houthakker  and  Markowitz,  involves  manipu¬ 
lation  of  matrices  whose  sizes  are  independent  of  the  number  of  linear  inequalities 

defining  the  polyhedral  region,  in  contradistinction  to  methods  based  on  the  simplex 

3  4 

algorithm,  such  as  those  of  Beale  and  Frank  and  Wolfe,  in  which  the  sizes  of  the 
matrices  increase  linearly  with  the  number  of  inequalities.  This  is  one  of  the  princi¬ 
pal  reasons  for  considering  algorithms  alternative  to  ones  based  on  the  simplex 
algorithm. 

Denote  the  polyhedral  region  by  U,  defined  by  the  inequalities 

a^  'x  >  bj,  i  *  1,  . , . ,  p,  (1.  1) 
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where  x'  *  (Xj, ....  xfl)  is  a  vector  variable,  and  a.'  *  (a.  y  . . . ,  a^n>,  bj,  i  *  1, . .  .  ,p 
are  a  set  of  vector  and  scalar  constants,  respectively.  The  Xj’s  are  not  necessarily 
non-negative;  however,  if  they  are,  these  conditions  are  presumed  included  in 
(1.  1).  Since  any  equality  can  be  written  as  two  inequalities,  then  (1.  1)  also 
includes  any  equality  conditions.  The  quadratic  is  given  by: 


Q(x)  * 


S  t 

J*1  h=l 


(1.  2) 


where  E  *  (r^}  is  a  positive  definite  n  xn  matrix.  Since  any  minimization  problem 
of  the  type  under  consideration  is  specified  once  the  inequalities  (1.  1)  and  the 
quadratic  Q(x)  are  given,  a  problem  will  be  defined  to  be  a  pair  (Q,  U)  where  Q  is  a 
quadratic  of  the  form  Eq.  (1.  2),  and  U  is  a  polyhedral  region.  A  solution  to  a 
problem  will  be  a  point  in  U  minimizing  the  quadratic. 

If  the  region  U  contains  the  origin,  then  this  point  is  the  solution;  otherwise  it 
is  known  that  the  solution  is  on  the  boundary  of  U  and  elementary  calculus  methods 
cannot  be  used  to  determine  this  point. 

The  general  method  used  here,  and  by  Houthakker  and  Markowitz,  is  to  start 
with  some  point  known  to  be  in  U  and  to  formulate  a  problem  whose  solution  is  this 
particular  point,  then  to  continuously  deform  the  problem  into  the  original  one  while 
keeping  track  of  the  solution.  Houthakker  and  Markowitz  do  this  by  means  of  an 
additional  inequality  1  x  >  b^_.  where  Vi  is  a  parameter  with  the  range  -» 
to  «.  This  inequality  bisects  U  into  two  parts:  Uj  =  U rv { x  :  a^^  'x  >  b^j}  and 
Ug  *  U  -  Uj  (one  of  these  may  be  empty).  For  some  value  of  bp+l’  Uj  will  consist 
of  a  single  point  (or,  in  exceptional  cases,  a  hyperplane).  The  minimum  ofQ(x) 
over  this  particular  Uj  is  easily  found.  As  b^  1  is  then  decreased  (or  increased), 
Uj  grows  and  the  solution  in  U  ^  moves  continuously  along  the  surface  of  U  to  the 
solution  over  U.  Since  U  is  a  polyhedron,  each  point  on  its  surface  can  be  charac¬ 
terized  as  the  intersection  of  a  set  of  hyperplanes  a.  'x  *  b^,  i  c  I,  where  I  is  a 
subset  of  the  integers  1, . . . ,  p.  Thus,  the  solution  can  be  kept  track  of  by  a  set 
function  Kb^ j),  which  is  a  function  of  the  parameter  b^j,  since  the  solution  is  a 
function  of  b^j.  The  accompanying  algorithm  is  principally  concerned  with  rules 
for  determining  when  to  add  or  delete  indices  from  Kb^j). 

Markowitz  used  this  method  in  connection  with  what  he  called  the  portfolio 
problem,  in  which  the  equation  a^j  'x  *  b^  ^  has  a  special  interpretation,  and  the 
solutions  for  all  values  of  b^j  are  of  interest.  Therefore,  the  method  seems  quite 
well  suited  to  the  application. 

In  the  method  used  here,  one  begins  with  a>  subproblem  of  the  original  problem, 

which  is  easy  to  solve,  and  obtained  by  setting  x„  *  . . .  =  x  =  0  and  replacing  Q(x) 

^  n  — 
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by  the  rank  one  quadratic  x.  r . .  Then  one  permits  x,  to  be  non-zero  and  deforms 

2  *  ^  i  n  o  * 

Xj  r^j  into  the  rank  two  quadratic  Xj  rjj  +  x2  r22  +  2xlx2r12  *n  8UC**  a  w*y  the 
solution  moves  continuously  over  U.  The  solution  then  becomes  a  function  of  t,  and 
is  determined  explicitly.  Thus  its  value,  if  known  at  the  beginning  of  the  deformation, 
will  be  known  at  the  end.  This  is  continued:  the  variables  x.  are  introduced  one  at 

Sk-1  B 

i  j  =  l  xirijxj  *8  Reformed  in*0  the  quad¬ 
ratic  one  rank  higher,  until  one  'returns  to  the  original  quadratic 

Q(x).  The  deformation  of  the  quadratic  is  achieved  by  replacing  the  elements  of 
R  by  expressions  tr^  and  t  +  r^  where  t  is  a  parameter  taking  values  in  an  ap¬ 
propriate  interval. 

Along  the  way  to  the  final  solution  the  solutions  to  the  subproblems  are  obtained. 
These  solutions  are  needed  in  many  applications.  In  particular,  n,  the  number 
of  variables,  may  Itself  be  a  variable  and  the  minimum  value  of  the  quadratic  a 
function  of  n.  to  be  determined.  For  such  problems  our  method  seems  the  more 
natural  one. 

The  addition  of  a  linear  term  to  the  quadratic  involves  a  minor  modification  of 
the  algorithm  (in  principle,  a  suitable  linear  transformation  will  eliminate  the  linear 
part),  and  since  a  second  degree  polynomial  can  be  continuously  deformed  into  its 
linear  component,  any  linear  programming  problem  can  be  solved  by  embedding  it 
in  a  second  degree  polynomial  programming  problem,  solving  the  latter,  and  then 
deforming  the  second  degree  polynomial  into  its  linear  component. 

The  application  which  led  to  this  method  is  the  problem  of  linear,  minimax  pre¬ 
diction  of  deterministic  processes  using  a  quadratic  Iobs  function.  In  this  application 
*  . . . ,  xn  are  the  coefficients  of  the  linear  predictor,  while  R  is  the  covariance 

matrix  of  the  observations.  The  observations  have  the  form  y^(9)  +  Zy  j  *  1, ....  n. 


a  random  variable. 

Subject  to  certain  restrictions,  it  can  be  shown  that  the  problem  of  determining 
the  x^'s  reduces  to  a  quadratic  program-, ning  problem  with,  in  general,  an  infinite 
number  of  linear  inequality  restrictions.  In  applications  one  can  sometimes  obtain 
a  good  approximation  to  the  problem  with  a  finite,  though  large,  number  of  inequali¬ 
ties  -  hence  the  need  for  a  quadratic  programming  algorithm  which  can  handle  large 
numbers  of  inequalities. 

In  this  application  it  is  generally  desirable  to  obtain  the  predictors  based  on  all 
sample  sizes  up  to  n,  or  to  let  n  be  a  variable  of  the  algorithm  and  stop  according 
to  some  rule  involving  the  expected  loss. 

The  theory  on  which  this  algorithm  is  based  is  given  in  the  next  two  sections. 

In  Section  2  some  definitions  and  preliminary  facts  stated  as  propositions  are  given. 
The  algorithm  is  more  directly  based  on  three  propositions  given  in  Section  3.  They 
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mre  called  propositions  rather  than  theorems  for  lack  of  a  more  suitable  word  to 
describe  some  highly  specialized  results  in  the  theory  of  linear  spaces  and  convex 
functions. 

The  basic  algorithm  is  described  in  Section  4,  and  an  example  given  in  Section 
5;  Sections  6,  7,  and  8  present  the  treatment  of  three  possible  types  of  degeneracies 
which  may  occur  in  the  basic  algorithm;  Section  9  treats  computing  procedures, 
while  in  Section  10  the  method*is  extended  to  linear  programming. 

2.  DEFINITIONS  AND  PRELIMINARIES 

A  problem,  as  previously  stated  in  Section  1,  is  a  pair  (Q,  U),  where  Q  is  a  quad¬ 
ratic  form  and  U  is  a  polyhedral  region  in  the  space  of  the  variables  of  the  quadratic 
form.  A  solution  to  the  problem  is  a  point  in  U  minimizing  Q.  It  will  be  assumed  that 
U  is  nonempty  and  does  not  contain  the  origin.  Let  I,  with  or  without  subscripts,  be 
any  subset  of  the  integers  1, .  . ,  p  and  let  hi  be  the  hyperplane  defined  by  the  equalities 
£i  *  bj,  i  f  1  Let  t  be  a  real  parameter  and  R(t)  3  {r^ft)}  be  an  n  X  n  matrix 
such  that  r.j(t)  is  either  constant  or  linear  in  t,  but  not  all  r^(t)  are  constant,  and 
such  that  R(t)  is  positive  definite  for  all  t  in  its  domain.  This  domain  will  be  either 
a  finite  or  infinite  interval  with  the  lower  bound  of  this  interval,  a,  always  finite  and 
the  upper,  0,  finite  or  infinite.  If  the  interval  is  finite,  then  o  <  t  <  f>\  while  if 
infinite,  then  a  <  t  <  p.  This  interval  is  also  denoted  by  T.  For  any  polyhedral 
region  U  of  x  space,  let  x(t,  U)  denote  the  solution  to  the  problem  (Q(x,t),  U) 
where  Q(x,  t)  3  x'R(t)x.  Thus,  x(t,  hi)  is  a  solution  to  the  problem  (Q(x,t),hl)  where 
hi  is  specified  by  a.  'x  =  bj,  i  e  I. 

Vectors  will  always  be  column  vectors  written  in  boldface  (indicated  by  under¬ 
lining).  For  any  sets  U,  V,  U  C  V  will  denote  U  properly  contained  in  V;  uC  V 
will  denote  UCV  or  U  =■  V.  To  reduce  the  amount  of  notation,  arguments  of  functions 
will  be  omitted  where  the  functional  dependence  is  clear  from  the  text. 

The  polyhedron  U  is  convex  and  so  is  any  hyperplane  hi  Moreover,  Q(x,t)  is 
a  strictly  convex  function  of  x,  since  Q(x,  t)  is  positive  definite.  Therefore,  x(t,U) 
exists  and  is  unique  for  t  c  T  and  similarly  for  x(t,hl),  if  hi  is  nonempty. 

For  any  t  e  T  let  I(t)  be  the  set  of  all  integers  i,  1  <  i  <  p,  such  that  a^  'x(t,  U)  * 
b. .  Then, 

x(t,hl(t))  3  x(t.U),*  tfT,  (2.1) 


♦  Proof;  x(t,U)  is  inhl(t),  and  hl(t)  is  contained  in  any  supporting  hyperplane  (that  is, 
any  hyperplane  of  the  form  a'x  «  b  such  that  a'x  3  b  for  all  x  e  U,  with  equality  hold¬ 
ing  for  some  x  e  U)  to  U  at  the  point  x(t,  U).  "Now,  since  x(T,  U)  minimizes  Q(x,t)  over 
U,  it  follows  that  the  ellipsoid  consisting  of  all  x  such  that  Q(x,t)  3  Q(x(t,  U),f)  does 
not  intersect  U  except  at  the  point  x(t,  U).  Since  both  U  and  tf3s  ellipsoid  are  convex. 
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•o  that  an  explicit  formula  for  x(t,  U)  la  given  by  the  formula  for  x(t,hl(t))  .  Note 
that  l(t)  la  maximal  with  reaped  to  its  given  property. 

The  formula  for  x(t ,  hi  (t»  ia  obtained  by  means  of  the  Lagrange  multiplier 
method;  that  ia,  the  unrestricted  minimum  of  Q(x,t)  -  2  J  ^  X^la^  '2  *  ^  *  IM. 

with  respect  to  x  and  X  ia  found.  After  differentiating,  one  obtains  necessary  and 
sufficient  conditions  for  a  minimum; 

R(t)x  -  A'X  ■  0  (2.  2) 


Ax  ■  b,  (2.  3) 

where  A  is  the  matrix  with  rows  a^' ,  k  «  I(t).  By  assuming  that  A  has  full  row  rank; 
that  is,  the  rows  of  A  are  linearly  Independent,  the  solution  is: 


x  -  x(t,  hi  (t))  ■  R'X(t)  A'& 


(2.4) 


X  *  X(t„  hi  (t»  •  (AR_1(t)  A')'1  b.  (2.5) 

Dired  use  of  this  formula  for  computation  would  be  very  inefficient;  a  better 
method  of  computing  x  and  X  is  given  in  Sedion  9.  However,  it  is  satisfactory  for 
expository  purposes.  If  A  is  nonsingular,  then  (2.  3)  has  a  unique  solution 
independent  of  R(t).  In  this  case  x  will  not  depend  t»i  t,  while  X  continues  to  do  so 
(except  if  b  *  0,  which  is  excluded  by  the  hypothesis  that  U  does  not  contain  the 
origin). 

For  fixed  A  and  b  the  coordinates  of  x  and  X  are  rational  in  t.  In  this  case  a  0 
'value  for  the  j'th  coordinate  of  X  has  the  interpretation  that  for  that  particular  value 
oft,  the  j'th  condition,  a^  'x  *  bj,  is  redundant.  More  precisely, 


Let  f(x)  be  some  fundlon  whose  partial  derivatives  exist  for  all  x 
and  let  ^  *  (Xj, . . . ,  $n)  be  a  stationary  point  of  f(x),  subject  to  the  k  +  r 
consistent  and  linearly  independent  linear  restridions,  a^  'x  *  bj,  i  *  1,  S  (2.  6) 
....  k+r.  A  necessary  and  sufficient  condition  that  £  is  also  a  stationary! 
point  of  f(x)  subjed  to  just  the  first  k  linear  restridions  is  that  j  ■ 

. . .  *^k+r  «  0,  where'Xj,  ....  ^k+r.  the  Lagrange  multipliers,  are  a 
solution  to  the  equations 


there  is  a  separating  hyperplane,  H,  (a  hyperplane  which  is  supporting  hyperplane  to 
both  U  and  the  ellipsoid),  containing  the  point  x(t,  U)  and  hence,  containing  hl(t).  As 
the  minimum  of  Q(x,t)  over  H  is  at  x(t,  H),  the~latter  point  also  minimises  Q(x,t)  over 
hl(t). 


For  any  I,  the  coordinates  of  x(t,hl)  are  rational  in  t,  hence  x(t,hl)  is  continuous. 
Furthermore,  x(t ,  hl(t)>  is  also  continuous  in  t.  This  is  true  because  Q(x,t)  is 
continuous  in  x  and  t,  hence  minX£^Q(x,t)  *  Q(x(t,hl(t»  ,t)  is  continuous  int.  Since 
xft.hHt))  is  unique,  it  is,  therefore,  continuous. 

The  following  section  establishes  the  properties  of  I(t)  on  which  the  algorithm 
is  based:  as  t  traverses  the  interval  T  in  either  direction,  l(t)  makes  only  a  finite 
number  of  changes  in  value  and  each  change  involves  one  or  more  (generally  one) 
additions  or  deletions  to  the  set  I(t).  At  the  points  t  where  these  additions  or 
deletions  occur,  certain  conditions  hold  which  enable  us  to  determine  where  these 
points  are  and  which  indices  to  add  or  delete  from  I(t). 

3.  THEORY  OF  THE  AUQORITH1C 

For  each  l£{l . .  p}  and  j  i  I  there  are,  at  most,  a  finite  number  of  points  of 

intersection  of  x(t,  hi)  and  {x:  au  1  x  •  b^}  for  t  <  T;  hence,  the  total  number  of  points 
of  intersection  for  all  pairs  fl,  j),  j  i  IQ  {I, ...  ,p}  is  also  finite.  Lets*  be  the  set 
of  all  these  points  of  intersection.  Then  at  any  point  t  where  I(t)  changes  value, 
necessarily  x*  ■  x(t*  ,hl(t* ))  «  S*  .  Now  it  will  be  shown  that  to  each  x*  e  S*  there 
correspond,  at  most,  two  points  t  of  change  of  I(t),  Implying  that  I(t)  changes  value 
at  no  more  than  a  finite  number  of  points  in  T.  If  there  are  more  than  two  points, 
then  there  exists  t1  <t2  <t3  such  that  x*  *  xftj.  hl(tj))  *  xftg.hKtg))  i*  x(t2>  hl(t2))  * 

*  Proof:  The  sufficiency  is  immediate.  For  the  necessity,  if  It  is  a  stationary  point 
of  f(x)  subject  just  to  at  'x'bj,  i  *  1, . . . ,  k,  then  there  exist  ${7 ...  satisfying 

-S5j-|f<*>+  £  \  iiij  "°-  1  "  1 . 

Subtracting  the  latter  from  the  equation  in  (2.  6)  for  J»l, . . , ,  n,  one  obtains: 

^  (^1  -  Mi>  aj  +  J  aj  ■  0. 

1*1  i»k+l 

Since  the  a<  's  are  linearly  independent,  ft  ■£*,  1  *  1, . . . ,  k,and  ft  «  0,  i  «  k+1, . . . ,  r. 
Q.E.D. 
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Xj.  Since  Q(x,t)  le  strictly  convex  in  x  for  fixed  t,  Q(x*  ,tj)<  Q(x2  ,  tj),  Q(x*  >t3)< 
Q0c2.t3),  end  Q(x2>t2)<Q(x*  ,t2).  On  the  other  hand,  ae  the  elementa  of  R(t)  are 
linear  in  t,  Q(x,t)  ia  linear  in  t  for  fixed  x.  However,  the  laet  three  inequalities 
.cannot  be  true  simultaneously,  ae  the  reader  may  readily  verify  with  a  aketch. 
Geometrically,  thia  implies  that  x(t ,  hl(t))  cannot  describe  a  loop. 

Call  any  point  t,  where  I(t)  changes  value,  a  transition  point  and  the  change 
itself  atransition.  For  definiteness  take  t  as  decreasingin  T*  [a, /l) .  At  a  transi¬ 
tion  point  ^  there  are  three  possible  types  of  transitions:  (1)  either  one  or  more 
subscripts  are  added  to  I(t);  (2)  one  or  more  are  subtracted:  (3)  both  subtraction 
and  addition  take  place. 

If  Type  (1),  then  I(tj+)C  I(tj)  ■  I(tj-);  the  alternative,  I(tj+)  *  I(tj)C  Ktj-)  can¬ 
not  occur  because  xft^.hlft^))  e  hl(tj-)  and,  by  definition,  I(t)  is  always  maximal.  For 
a  similar  reason  in  Type  (2),  I(tj+)  ■  I(tj)3l(tj*)  and,  in  Type  (3),  Ktj+lC  Iftj)  D 
I(tj-).  In  the  latter  case,  there  are  two  transitions  at  a  single  transition  point. 

Let  be  the  sequence  of  values  which  I(t)  takes,  and  t0 . t  ,  the 

i  in  a  m- 1 

sequence  of  transition  points  as  t  decreases  in  the  interval  p  >  t  >  a.  In  order  to 
have  the  index  for  the  transitions  coincide  with  the  index  for  the  transition  points  take 
tj  •ti+1  if  Type  (3)  above  occurs  at  tj.  Noting  that  at  t^  the  transition  Ijj  -  Ij  takes 
place,  we  have 

I(t)  •  It  for  tt  >  t  >  ti+1  if  tt  >  ti+1 

I^i>  "  *i-l  **  indices  are  added  to  I(t)  at  ^  (Types(l)  or  (3))  ,  (3  j) 

■  Ij  if  Indices  are  subtracted  from  I(t)  at  tj  (Type  (2». 


It  is  convenient  to  extend  the  above  results  to  the  endpoints  a  and  p.  If  tj  * 

P  <  «.  then  define  I(tj)  ■  I0>  In  general,  I(tj)  ■  I(tj-)  *  Ij,  so  that  IQ  may  equal  Ij. 
Similarly,  define  Ha)  ■  I(tm)  ■  Im+1,  where  Im  may  equal  I^j.  If  p  *  •»  and 
llm^^xft,  hlj)  >  x(«>,  hlj)  exists, then  Kt^)  will  also  exist.  It  is  assumed  below  that 
x(«,  hlj)  and  Htj)  exist  where  they  are  used.  In  the  following  section  their  existence 
is  proved  for  the  particular  algorithm  developed. 

For  use  in  the  sequel  note  a  few  conclusions  which  follow  from  these  definitions 
will  now  be  given.  There  may  not  be  more  than  two  adjacent  equal  transition  points 
and,  on  the  other  hand,  no  adjacent  sets  Ii>  Ii+1  which  are  equal.  Slightly  less 
elementary  is 


*1+1  >  *1+2 


if 


*i^  !i+l 


*i  >ti+i  if  H^i+i. 


(3.  2) 
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Proof.  If  It  D  Ii+1,  the  I(ti+1)  ■  IJ+ j  by  (3.  1).  If  tj  -  tj+1. 
then  the  top  conclusion  follows  immediately.  On  the  other  hand,  if 
tj  >t1+1  *  ti+2>  then  Ij  *  I<t1+1+)C  I(tl+1)  *  Ij+1  which  contradicts 
the  hypothesis.  The  second  statement  of  (3.  2)  is  proved  similarly. 

A  third  conclusion  from  these  definitions  is 

xftj.hlft.))  t  hlj.j  r\  hlj,  (3.3) 

since  tj  is  the  point  of  transition  of  Ij_  j  to  Ij  and  x(t,  hl(t))  ,  being  continuous,  must 
satisfy  Sj  'x  (tj.  hl(tj))  «  b^,  j  t  Ij.jU  Ij.  Hence,  xft^hHt^))  e  h(Ij_j  W  Ij)  * 

Mi-l^“i 

These  definitions  and  results  are  given  in  terms  of  decreasing  t;  for  t  increas¬ 
ing,  obviously  *i  -  *1+1'  which  affects  (3.  2)  and  a  number  of  other  statements. 

In  addition  t+  and  t-  must  be  interchanged  wherever  they  appear.  Throughout  this 
section  t  will  continue  to  be  taken  as  decreasing. 

As  t  decreases,  x(t,hl(t))  describes  a  continuous  path  on  the  surface  of  U.  If 
Ij  3li+1  at  the  transition  point  *i+l'  then,  geometrically,  x(t.hl(t))  leaves  the 
hyperplane  hlj  which  may  be  regarded  as  an  edge  of  U  and  moves  in  the  higher  dimen¬ 
sional  hyperplane  hL+1  which  may  be  regarded  as  a  face  relative  to  hlj.  By  (2.6) 
it  follows  that  at  the  point  tj+j,  where  the  transition  h-h+l  occurs,  the  Lagrange 
multipliers  Xjft^j.hlj)  corresponding  to  the  conditions  a^  'x  *  bj,  j  t  Ij-li+j,  become 
zero.  This  is  a  necessary  condition  satisfied  by  the  triple  tj+1,  Ij,  andli+j. 

On  the  other  hand,  if  IjClj+1  at  tj+j,  then  a^  'x(tj+1>  hlj)  -  *  0  for  j  e  Ij+1 

-  Ij,  which  Is  a  necessary  condition  satisfied  by  the  triple  t^j,  Ij,  and  IJ+1  in  this  case. 

Thus  a  necessary  condition  for  a  transition  of  either  type  at  the  point  t  ^  tj  is 
either 


Xj(t,  hlj)  «  0  for  some  j  e  Ij 


or 


(3.4) 


Sj'xft.  hlj)  -  bj  «  0  for  some  j  i  Ij  .  (3.  5) 

Neither  of  these  conditions  ulone  is  sufficient  for  its  respective  type  of  transition. 

In  particular,  there  may  exist  subsets  J',  J"  such  that  J*  Cl.  C  J",  and  points  t', 
t"  <tj  such  that  the  triple  t',  Ij,  and  J'  satisfies  (3.  4)  for  all  j  e  Ij-J,  while  the 
triple  t",  Ij,  and  J"  satisfies  (3.  8)  for  all  j  «  J*  -  Ij.  The  transition  point 
*1+1*  if  it  is  one  of  the  two  points  t' ,  t" ,  by  definition,  must  be  the  larger  of  the  two 


1 


l 
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(the  smaller  if  t  is  increasing).  Hence,  any  sufficient  conditions  must  take  account 
of  both  (3.  4)  and  (3.  5)  (this  explains  the  apparent  circularity  of  Props.  1  and 
2,  below).  Furthermore,  the  occurrence  of  (3.  4)  or  (3.  5)  is  not  sufficient  for 
a  transition  (of  either  type)  since  the  multipliers  can  vanish  at  a  non-transition  point. 

The  following  two  propositions  give  sufficient  conditions  for  the  two  types  of 
transitions,  l^C.  and  1^  2>  Ii+1<  respectively.  The  conditions  in  the  first  are 
also  necessary. 

Proposition  1.  Let  J  be  a  subset  of  the  integers  1, ....  p  such  that  J  5  Ij*  i  *  1, 

. . . ,  m-1,  and  assume  that  t  is  a  decreasing  parameter.  Necessary  and  sufficient 
conditions  that  a  number  t'  and  the  subset  J  are  tJ+1  and  Ij+j,  respectively,  are 

ft)  tt  >  t'  >  « 

fli)  'x(t\  hlj)  »  b,.  j  c  (J-Ij) 

(iii)  J  is  the  maximal  subset  satisfying  (ii)  at  t' 

(iv)  t'  is  the  largest  number  satisfying  (i)  -  (iii)  for  some  J  '  15  I., 
where  J  '  is  a  subset  of  {1, ....  p> 

(v)  The  conditions  (i),  (iii),  and  (iv)  of  Prop.  2  are  not  satisfied 
for  a  number  t"  >  t' . 

Proof:  We  first  prove  the  necessity  of  these  conditions,  i.  e. , 
that  t  '  •  ti+1  and  J  »  Ii+1  satisfy  (i)  -  (v)  if  Ii+1 

B:  definition,  tj  >  ti+1>  If  t^  »  tJ+1  then,  again  by  definition,  ti+1  ■ 
ti+2  and,  by  (3.  1),  it  follows  that  It  ■  I(tt)  »  I(tl+l),  while  Ii+1  * 

I(ti+1-).  But  x[ti+1,  M(ti+1>]  e  hli+10  hlj  C  hli+1  by  (3.  3);  there¬ 
fore.  I(ti+1)  DIj  »  I(ti+1),  implying  that  I(ti+1)  *  li+1  and  con¬ 

tradicting  (3.  2).  Thus  tj  1 11+1,  proving  (1). 

By  definition  of  the  transitions  and  transition  points  (ii),  (iii),  and  (iv) 
are  true.  For  (v),  if  there  exists  t"  and  J',  t"  >  t^+j,  and  J  'Clj,  satisfying 
(i),  (ii).  and  (iv)  of  Prop.  2,  then  Q(x(t,hJ  *),t)  <  Q(x(t, hlt), t)  in  t"  > 
t  >t"  -  «'  >  t|+|,  for  some  *'  >  0  by  (iv)  of  Prop.  2,  and,  by  (ii) 
of  Prop.  2,  equality  cannot  hold  identically  throughout  the  interval. 

But  inequality  contradicts  the  definition  of  1^;  hence,  it  follows  that 
(v)  is  necessary. 

Now,  from  (iii)  and  (iv)  of  the  present  proposition  it  follows  that 
t^+1  and  Ii+1  are  the  unique  pair  satisfying  tt)  -  (v);  therefore,  the 
conditions  are  also  sufficient.  Q.  E.  D. 

Proposition  2.  Assume  that  the  restrictions  a^  'x  «  j  c  Ij,  are  linearly  independ¬ 
ent,  and  t  is  decreasing.  If  there  exist  t'  and  J,  JC  Ij.  0  <  i  <  m-1,  such 
that  (i)  -  (vii) below  are  satisfied,  then  t'  ■  tJ+1  and  J  *  Ij+j. 
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(I)  tt  >  t'  >  a 

(II)  X.ft'.hlj)  «  0  tor  J  «  lj  -  J.  and  there  exists  at  least  one  j  e  Ij  -  J  such 
that  Xjft.hlj)  i  0  In  t 

(Hi)  Sj  'x(t,hJ)  -  b .  i  0  tor  j  e  It  -  J 

(iv)  x(t,hJ)  e  U  for  t*  >  t  >  t'  -  tor  some  <’  >  0 

(v)  J  is  the  unique  smallest  subset  of  I.  satisfying  (i)  -  (iv) 

(vi)  t1  is  the  largest  number  satisfying  (i)  -  (iv)  tor  some  proper  subset  of 
!i 

(vii)  conditions  (i),  (ii),  and  (Hi)  of  Prop.  1  are  not  satisfied  for  some  t" 

>  t'  and  J'  5  lj. 

Proof:  We  first  prove  that  t'  >  tj+j,  then  the  converse.  Now,  either 
*i+l  ^  *i  or  *i+l  s“  V  the  former,  and  tJ+1  >  t',  then  Ii+1  and  ti+J 
satisfy  (i),  (ii),  and  (iii)  of  Prop.  1.  violating  (vi)  of  this  proposition: 
hence,  t'  >  tj+1  if  lj- 

If,  on  the  other  hand,  I,+1  C  lj,  then  it  will  be  shown  that  t.+1 
and  I1+1  satisfy  (i)  -  (v);  hence,  by  (vi),  it  will  follow  that  t'  >  t.+J. 

By  definition,  (i)  and  (iii)  are  true  while,  by  <3.  2),  tJ+1  >  ti+2- 
so  that  t1+1  and  IJ+1  satisfy  (iv),  again  by  definition.  By  (2.  6), 

Xj(ti+1,hlj)  »  0  for  j  <  lj  -  I.+1;  moreover,  the  set  of  multipliers 
Xj,  j  c  1^  -  cannot  all  be  identically  0  with  respect  to  t  for,  if 

they  are  then,  in  an  interval  t1+1  >  t  >  tj+1  -  c  >  tJ+2,  e  >  0  (which 
exists  by  (3.  2))  lj  rather  than  I^+  j  is  maximal,  contradicting  the 
assumption  that  t1+J  is  a  transition  point.  Thus,  ti+1  and  I.+]  satisfy 
(ii).  Finally,  if  (v)  is  not  true,  then  the  strict  convexity  of  Q(x,  t)  with 
respect  to  x'  implies  that  x(t,hli+1>  does  not  minimize  Q(x,t)  in  the 
interval  tl+1  >  t  >  tJ+1  -  c'.  Thus,  tJ+1  and  Ii+1  satisfy  (ii). 

Therefore,  under  either  assumption  Ii+1C  lj  or  *i  ^1i+l'  *’  -  H+l 
and,  since  these  assumptions  are  mutually  exhaustive,  it  follows  that 
t'  >  tj+1  without  either  assumption.  If  now,  t'  >  t^+1  then,  since 
x(t,hl{)  is  the  unique  minimum  of  Q(x,t)  over  U  in  the  interval  t'  > 
t  >  tJ+1  and,  since  J  Oj  implies  Q(x(t,hJ),t)  <  Qlxft.hl^.t)  in 
this  interval,  then  it  follows  from  (iv)  that  x(t,hJ)  *  x(t,hlj)  in  a 
subinterval  of  this  Interval.  But,  since  Xft.hl^)  is  rational  in  t,  it 
follows  by  Eq.  (2.  6)  that  Xj(t,hlj)  ;  0  for  j  «  lj  -  J,  contradicting  (ii). 
Therefore,  t'  <  tj+j  and,  together  with  the  previous  result,  it  follows 
that  t'  *  ti+1- 

It  remains  to  show  that  J  =  Il+1-  First  we  prove  that  Il+1C  lj* 

If  tJ+1  *  t'  ■  tj,  then  condition  (i)  of  Prop.  1  is  violated:  hence,  I1+ j 
It-  The  only  alternative  is  *1+1^  lj.  On  the  other  hand,  if 
tj  >  ti+l  then  (i)  of  Prop.  1  is  satisfied.  Now  we  show  that  if  IJ+ jZ5lj, 


n 


then  (ii)  and  (iii)  are  aleo  satisfied,  thus  violating  (vil)  of  the  present 
proposition.  Thus  the  conclusion  again  will  be  Ij+jC  Ij-  By  (3.3) 
x  (ti+1.hl(ti+1»  chl^nhlj  Therefore,  since  I(ti+1)  is  maximal, 

I(ti+1)  2  ,i'JIi+r  But-  by  <3-  D,I(ti+1)  equals  either  It  or  Ii+1> 
hence,  I(tj+1)  *  IjU  I^j.  Therefore,  if  I.+1  D  Ij.  then  I(ti+1)  *  Ii+1, 
and  ti+1  and  Ii+1  satisfy  (ii)  and  (iii)  of  Prop.  1. 

Thus  Ij+J  C  Ij-  Now  if  Ii+1  3  J,  then  (iv)  is  violated  while,  since 
I [i )  -  (iv)  are  necessary  for  Ij+1>  if  Ii+1  ^  J,  then  (v)  is  violated.  There¬ 
fore,  Ii+1  *  J.  Q.  E.  D. 

The  conditions  of  Prop.  2  are  only  sufficient  because  it  is  possible  for  ti+1  and 
IJ+1  to  satisfy  all  the  conditions  except  (v).  This  is  illustrated  in  Figure  1,  for 
which  the  following  assumptions  are  made:  hli+1  coincides  with  an  edge  of  U  to  the 
right  of  the  vertex  xft^j.hlj);  x(ti+1,hlj)  *x(ti+1,hJ);  x(t,hli+1>  moves  along  hlj+1 
in  the  direction  of  the  arrow  as  t  decreases;  x(t,  hJ)  moves  along  the  dashed  line 
in  the  direction  of  the  arrow  as  t  decreases,  but  the  dashed  line  is  outside  U.  Then 
Ii+1  will  satisfy  all  the  conditions  except  (v)  of  Prop.  2  while  J  will  satisfy  (v),  but 
not  (iv).  This  contingency  will  be  called  a  degeneracy  of  Type  3,  one  of  three  possible 
types  which  are  treated  in  a  similar  manner;  namely,  by  detouring  around  the  region  in 
x  space  where  the  difficulty  occurs,  in  this  case  around  the  point  x(ti+ j,  hlj).  This 
degeneracy  is  treated  in  Section  8. 


Figure  1. 

Proposition  2  excludes  the  case  where  the  linear  restrictions  corresponding  to 
Ij  are  linearly  dependent.  It  is  very  desirable,  for  computational  reasons,  that  the 
restrictions  be  linearly  independent.  The  contingency  which  arises  when,  for  some 
transition  point  tj,  the  rows  a^',  j  e  I(tj),  are  linearly  dependent  is  called  a  degener¬ 
acy  of  Type  2  which  is  treated  in  Section  7. 

If  t'  •  tj  in  Prop.  2  then  there  results  a  double  transition  at  the  point  t^.  This 
may  occur  in  two  ways,  although  they  are  both  generally  rare  events.  Figures  2  and 
3  illustrate  these  double  transitions.  The  straight  lines  and  surfaces  cut  off  by  these 
lines  indicate  hyperplanes  (the  lines  being  the  intersections  of  the  surfaces),  while 
the  trajectory  of  x(t,  U)  is  indicated  by  the  straight  or  curved  lines  with  arrows. 
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In  Figure  2  x  is  tangent  to  a  hyperplane  at  the  point  t..  This  tangency  is  equivalent 
to  aj  ’x(t,hlj)  -  bj  *  0  having  a  double  root  at  t(  =  t.+  1  for  j  t  !J+1  -  Ij.  In  Figure  3 
x  undergoes  an  abrupt  change  of  direction  at  t.,  touching  the  hyperplane  hlj+ j  for 
just  the  single  parameter  point  tj  =  t.+  j.  This  event  is  equivalent  to  'x(t.+ j, hL)  - 
bj  3  0,  j  e  Ij+1  -  Ij,  and  j ,  hi. )  =  0  for  one  or  more  k  <  I(.  It  neither  of  the 
two  events  illustrated  by  the  figures  occurs,  it  can  be  assumed  that  t.  >  t.+ j. 

Condition  (iii)  is  added  to  Prop.  2  only  because  the  1^  have  been  defined  as  maxi¬ 
mal.  From  the  proof  of  (2.  6)  it  follows  that  if  one  first  finds  J  ignoring  (iii), 
then  the  addition  of  any  subscripts  to  J  in  order  to  satisfy  (iii)  will  not  affect  the  X's 
nor,  of  course,  x. 


4.  THE  GENERAL  FORM  OF  THE  ALGORITHM 

Recall  that  a  problem  was  defined  to  be  a  pair  (Q(x),  U)  ,  where  Q(x)  is  a  quad¬ 
ratic  form  in  x,  and  U  is  a  polyhedral  region  in  /  space.  The  algorithm  consists  of 
n  stages;  in  the  k'th  stage,  1  <  k  <  n,  the  original  problem  is  solved  subject  to  the 
additional  restrictions  xk+J  «...  =  xfl  *  0  (since  these  can  be  written  as  2(n  -  k) 
inequalities,  a  new  polyhedral  region  is  obtained,  hence  a  new  problem) .  The  stages 
are  linked  together  by  having  the  solution  to  each  stage  as  the  initial  solution  to  the 
succeeding  stage.  To  describe  this  more  precisely,  it  is  necessary  to  first  introduce 
some  terminology. 

If  (Q  (x,t),  U),  a  <  t  <  ft  is  a  family  of  problems  with  parameter  t,  t  taken  as 
decreasing,  then  the  initial  and  terminal  solutions  relative  to  this  family  of  problems 
are  defined  to  be  x(/3,  U)  and  x(a,  U).  Call  HP)  *  IQ  the  initial  restriction  set,  and 
1(a)  •  Im  the  terminal  restriction  set  relative  to  the  family.  If  t  is  taken  as  increas¬ 
ing  from  a  to  P,  then  the  designations  'initial'  and  'terminal'  are  reversed.  For 
brevity  the  designation  'problem'  for  'family  of  problems'  will  be  used. 

Let  equal 


13 


’  rl,k-l  0 


Vl,l . rk-l.k-l  0 


0  rkk+t. 


*nd  equal 


rl,  k-1  trlk 


rk-l,k-l  trj-l,k 


(4.  1.  1) 


. trk,k-l  rkk 


Note  that  In  (4. 1)  only  the  lower  right  element  depends  on  t,  while  in  (4.  1.  1) 
t  multiplies  every  element  in  the  right  column  and  lower  row  except  r^. 

The  k-l'st  stage  is  the  problem  of  minimizing  x1 1Rk_j(l)x, where  x'  *  (x^. . . , 
xk_1).  subject  to 


^  *ij*j  -  bi' 


«...  -xn-0. 


Aasuming  this  problem  has  been  solved,  the  elements  of  interest  for  the  k'th  stage, 
k  ■  2, . . .  ,n,  are  the  terminal  restriction  set  1(1)  and  the  terminal  solution  x(  l,hl(l» 
which  lies  on  the  hyperplane  hl(l).  The  k'th  stage  is  subdived  into  two  subproblems. 
The  matrix  of  the  quadratic  of  the  first  subproblem  of  the  k'th  stage  is  jR^ft)  with 
t  going  from  »  to  0,  and  the  polyhedral  region  is  defined  by 


t  Vj-bi* 


i  *  1, ....  p 


Vl*-  *  xn  *  °’ 


It  is  shown  below  that  the  terminal  solution  and  restriction  set  of  the  k-l'st  stage 
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equal,  respectively,  the  initial  solution  and  initial  restriction  set  of  the  first  sub¬ 
problem  of  the  k'th  stage.  It  follows  that  the  solution  is  continuous  between  stages. 

o 

At  the  conclusion  of  the  first  subproblem  the  term  x^  r^  has  been  added  to  the 
quadratic.  The  second  subproblem  adds  the  cross  product  terms  by  taking  gR^ft)  as 
the  matrix  of  the  quadratic  of  this  subproblem,  and  allowing  t  to  go  from  0  to  1, 
unlike  the  first  subproblem  where  t  went  from  «  to  0.  The  conditions  in  (4.  3) 
are  maintained  for  both  subproblems  of  the  k'th  stage.  It  is  shown  below  that  term¬ 
inal  and  Initial  solutions  and  restriction  sets  of  adjacent  subproblems  within  a  given 
stage  are  identical. 

Our  theory  makes  use  of  the  strict  convexity  of  the  quadratic  Q(x,  t),  with  respect 
to  x,  for  all  t  in  the  range  of  interest,  which  is  equivalent  to  the  positive  definiteness 
of  R(t).  By  hypothesis,  gR^l)  and  jR^it),  t  >  0,  are  positive  definite  for  k  *  1, ... , 
n,  while  from  2Rfc(t)  *  (l-tljR^iO)  +  tjR^U),  it  follows  that  .  ?■  ..ft)  is  positive  definite 
for  0<t  <  1,  k  *  2, . . . ,  n. 

The  identity  of  solution  and  restriction  sets  between  adjacent  subproblems  and 
stages  is  important  because  the  propositions  of  Section  3  only  provide  the  basis  of 
rules  for  adding  or  deleting  elements  to  the  set  I(t);  hence,  for  each  subproblem  and 
stage,  it  is  essential  that  we  have  an  Initial  restriction  set.  In  the  first  stage  an 
initial  restriction  set  is  unnecessary  because  the  problem  is  transparent;  a  solu¬ 
tion  can  be  found  almost  at  sight,  provided  a  solution  exists.  Non-existence  of  a 
solution  is  one  of  the  three  types  of  degeneracy  and  is  treated  in  Section  6. 

In  each  of  the  subproblems  into  which  the  k'th  stage  1b  divided  we  make  use  of 
Props.  1  and  2  to  traverse  the  associated  range  of  t  in  a  finite  series  of  jumps  from 
one  transition  point  to  the  next.  The  method  can  be  explained  most  clearly  by  an 
example  given  in  the  following  section.  ' 

Non-degeneracy  is  assumed  in  this  example.  Recall  that  the  three  possible  types 
of  degeneracy  are:  (1)  failure  of  a  solution  to  exist  in  the  first  stage,  which  is  equiva¬ 
lent  to  the  non-existence  of  any  x  satisfying  (4.  3)  for  k  *  1;  (2)  the  matrix  A  in 

(2.  2)  -  (2.  5)  does  not  have  full  rank,  so  that  x  and  X  as  defined  by  (2.  4) 
and  (2.  5)  fail  to  exist;  (3)  a  transition  of  the  type  Ij+  jC  which  fails  to  satisfy  condi¬ 
tion  (lv)  or  (v)  of  Prop.  2,  as  explained  in  Section  3. 

The  subdivision  of  the  over-all  problem  into  stages,  and  each  stage  into  sub¬ 
problems,  creates  a  chain  of  subproblems,  ordered  by  the  sequence  in  which  they 
are  solved.  This  section  is  concluded  with  a  proof  of  the  identity  of  terminal  and 
initial  solution  and  restriction  sets  of  adjacent  subproblems  in  this  chain. 

Proposition  3.  Assuming  non-degeneracy,  the  terminal  solution  and  terminal  restric¬ 
tion  set  of  any  subproblem  is  identical,  respectively,  to  the  initial  solution  and  initial 
restriction  set  of  the  following  subproblem. 

Proof:  The  proof  for  the  second  subproblem  of  the  k-l'st  stage  and 

the  first  subproblem  of  the  k'th  stage,  k  ■  2 . n,  will  be  given  first 


\ 


\ 

i 


15  . 


Throughout  this  paper  it  is  assumed  that  2Rn<l) ia  positive  definite, 
hence  so  is  jR^ft)  for  05*5“*  k  ■  2, . . . ,  n.  Therefore,  because  A, 
in  (2,  2)  and  (2.  3),  is  of  full  rank  by  our  non-degeneracy  assumption, 
it  follows  that,  for  all  t,  0  <  t  <  «,  there  exists  a  solution  xft)  =  (XjftX 
.  • . ,  *k(t»  .  given  by  (2.  4),  to  the  problem  determined  by  x'jR^tlx 
and  the  inequalities  (4.  3).  Then,  for  any  fixed  vector  jr'  *  (y^, 

•••  >  yk.v  O’)  satisfying  Eq.  <4.3),  this  solution  must  satisfy  x'(l)jR.  (t)x(t) 
<  y*!  R  k  <  t )  *  constant  <  oo .  Hence,  the  term  xfc2  (t)  (rkk  t)  in 
x'(t)  jRk<t)  x(t)  must  remain  bounded  as  t  -  •»,  implying  Ix^t)!  —  0 
as  t  -*  •».  Letting  x(«)  be  any  limiting  point  of  x(t)  as  t  -*  •  (at  least 
one  limiting  point  exists  since  x(t)  lies  in  some  bounded  subset)  it 
follows  that  x(*»)  satisfies  Eq.  (4.  2)  and  minimizes 


V  fl  xr  x 


(4.4) 


Any  solution  to  the  problem  defined  by  <4.  2)  and  (4.  4)  is  a  terminal 
solution  for  the  k-l'st  stage.  Furthermore,  since  (4.  4)  is  strictly 
convex  with  respect  to  <Xj . xk>,  x(*>)  is  unique. 

It  follows,  immediately,  that  I(»)  *  IQ  exists  and  equals  the  terminal 
restriction  set  of  the  k-l'st  stage. 

The  corresponding  result  for  the  pair  of  subproblems  within  a 
stage  follows  immediately  from  the  fact  that  jR^O)  •  j.R^(0).  Q.  E.  D. 


5.  EXAMPLE  l:  The  algorithm  when  no  degeneracies  are  present 

The  purpose  of  this  example,  and  the  other  examples  below,  is  to  exhibit  how 
the  algorithm  proceeds;  therefore,  to  avoid  distractions  from  the  main  ideas 
no  special  computational  techniques  are  used  for  the  solution  of  the  equations  which 
arise.  In  Section  9  some  computational  techniques  are  given  which,  it  is  believed, 
make  even  large  scale  problems  readily  accessible  to  present  day  automatic  computers. 

For  the  first  example 

Q(x)  »  3xj2  +  2x22  +  2XjX2  (5. 1) 


is  the  quadratic,and  U  is  the  region  defined  by  a^  'x  >  b^,  j  »  1, ....  6,  where  a^, 
bj  are  given  in  the  following  table; 
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over  U,  with  t  going  from  «•  to  0  and  with  { 1}  =  IQ,  the  initial  restriction  set.  The 
Lagrangian  is,  therefore. 


3xt2  +  <2+t)x22  -  2Xj  (Xj  +  2x2) 


from  which  we  obtain  the  conditions 


3Xj  -  Xj  «  0 


(2+t)  x2  -  2Xj  *  0 

holding  at  the  minimum.  Together  with  the  restriction,  Xj  +  2x2  =  4,  this  system 
of  equations  has  the  solution 

*1  *  t  +  14  *  x2  T+^4  ‘  X1  *  3  X1  • 


As  a  check  note  that,  as  t  —  Xj  -»  4  and  x2  -*  0,  as  required. 

Conditions  (3.  4)  and  (3.  S)  are  now  used  to  determine  the  first  transition 

point.  Condition  (3.  4)  is  not  satisfied  for  t  >  0.  Condition  (3.  5)  is  (4t+8)  a^  + 

24a.,  -  b,(t+14)  »  0.  To  find  the  values  oft  satisfying  this  for  J  /  1  one  solves 
js  1 

for  t,  and  obtains 


8ttil  -  24  *12 


Because  of  condition  (iv)  of  Prop.  1  it  is  desirable  to  consider  the  largest  value  of 
(S.  3)  as  the  first  candidate  for  a  transition;  this  is  t  =  10  for  j  «  2. 

As  the  conditions  of  Prop.  1  are  satisfied  for  t  *  10  and  J  *  {1,2}  they  are 
designated  t2  and  I2  respectively  (note  that  IQ  =  Ij  in  this  case).  Carrying  outthe 
minimization  of  Q(x,  t)  on  the  new  hyperplane  hl2  results  in 


Xj-2,  x2.l,  Aj-t-4,  X2.-t+10, 


where,  of  course,  x^  and  x2  are  independent  of  t  since  hl2  is  a  point.  Condition 
(3.  5)  is  inoperative  since  and  x2  do  not  depend  on  t.  From  (3.4)  Xj  * 

0  when  t  *  10.  Of  these  two  values  only  t  -  4  is  a  candidate  for  the  next  transition 
point  since,  by  the  remarks  following  Prop.  2,  t2  is  not  a  double  transition  point. 
On  the  other  hand,  the  result  t  «  10  serves  as  a  check  on  the  computations  since 
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t  *  10  is  precisely  the  value  of  t  for  which  the  condition  a2'x  *  b2  should  become 
redundant. 

Checking  the  conditions  of  Prop.  2  for  t  *  4  it  is  seen  that  all  the  conditions  are 
satisfied  except  possibly  (iv),  which  cannot  be  checked  at  this  point  in  the  computations. 
Therefore,  assume  that  (iv)  holds  and  continue,  reserving  the  check  for  later.  Tenta¬ 
tively.  t3  *  4,  I3  =  {2}.  On  hlg, 

X1  *  ~ tT'f  •  x2  *  TT5  *  *2  *  3xi* 
and  one  can  now  check  that  (iv)  of  Prop.  2  is  satisfied,  for 
(3t+6)an+  9aJ2-  (t+5)  bj>  0,  (5.4) 

for  t  *  4-.  If  (iv)  is  not  satisfied  then  a  degeneracy  of  Type  3  has  occurred  at  t  ■  4 
requiring  one  to  backtrack  to  deal  with  this  degeneracy. 

The  procedure  of  using  (3.  4)  and  (3.  5)  to  find  a  candidate  for  the  next  transi¬ 
tion  point  is  repeated.  (3.  4)  fails  to  be  satisfied  for  t  >  0,  while,  from  (3.  5), 


5b  -6a  -9a^ 


which  has  maximum  value  1  for  j  *  3.  By  Prop.  1  it  follows  that  t^  1  1  and  1^  * 
{2,3}. 

On  hi. 


v  „  3  ,  x  x  „  9  +  9t  .  .  3  -  3t 
X1  2  2'  X2  4  ’  *3  4 


As  neither  (3.  4)  nor  (3.  5)  is  satisfied  for  t  >  0  it  foUows  that  there  are  no  more 
transition  points  as  t  -  0;  hence,  at  t  *  0,  Xj  *  x2  *  3/2  iB  the  terminal  solution 
and  {2.  3}  is  the  terminal  restriction  set.  This  completes  the  first  subproblem  of 
the  second  stage. 

For  the  second  subproblem  take 

Q(x,  t)  *  3xj2  +  2x22  +  2tXjX2,  (5.  5) 

and  the  initial  restriction  set  {2,  3} .  This  time  t  goes  from  0  to  1.  We  obtain 


IS 


X1  *  x2  "  2 


.  9  +  6t 
2  '  4  * 


3 

3 


and,  as  neither  {3.  4)  nor  (3,  5)  is  satisfied  for  0  <  t  <  1,  one  concludes  that  no 
transitions  take  place  as  t  •*  1.  The  final  result  is,  therefore,  x^  ■  x2  «  3/2. 

This  simple  example  exhibits  all  the  steps  of  the  algorithm  when  no  degeneracies 
occur.  Note  that  it  is  not  necessary  to  vary  t  continuously,  even  though  we  spoke 
of  doing  this  in  the  theory;  instead,  the  transition  points  are  found  as  the  roots  of 
equations  linear  in  t.  In  general,  for  larger  problems,  second  degree  polynomials 
in  t  arise  after  the  second  stage,  but  the  roots  are  still  easily  found  by  the  well 
known  formula. 

The  following  section  describes  the  method  of  treating  the  degeneracy  occasioned 
by  the  lack  of  a  solution  to  the  first  stage. 


6.  DEGENERACY  OF  TYPE  1:  No  solution  to  the  first  stage 

Assume  that  the  first  stage  has  no  solution;  that  is,  there  is  no  Xj  satisfying 

a^Xj  >  bj,  j  ■  1, . . .  ,p.  To  start  the  algorithm  modify  the  original  problem  by 

adding  the  term  x  2  to  the  quadratic  and  x.  to  the  left  side  of  each  of  the  inequalities, 
®  2 

giving  a  new  quadratic,  xQ  +  x  'Rx,  and  a  new  region  defined  by 


x0  +  *j'i  >  bj*  . P-  (6.1) 

2 

The  first  stage  of  this  new  problem  has  the  quadratic  xQ  and  the  additional  conditions 

x,  «  x«  ■  0.  This  stage  has  a  solution  for  there  certainly  exists  an  x  satisfying 
*  «  2  ' 

(6.  1),  vis. ,  maxj  b^,  which  also  minimises  xQ  among  all  xQ  satisfying  the  re¬ 
straints. 

Therefore,  except  for  the  introduction  of  the  new  variable  xQ,  the  algorithm  is 
the  same  as  in  Example  1.  At  the  end,  though,  xQ  will,  in  general,  have  a  non-zero 
value,  and  to  obtain  the  solution  to  the  original  problem  xQ  must  be  driven  to  zero. 
This  is  done  by  replacing  xQ2  in  the  quadratic  by  (t+l)x02  and  letting  t  go  from  0  to 
•».  This  procedure  is  exactly  the  reverse  of  the  procedure  used  in  the  first  sub¬ 
problem  of  apy  stage,  where  a  variable  which  up  to  then  had  been  fixed  at  0  was 
allowed  to  become  non-zero.  The  procedure  is  exhibited  in  the  following  example. 
Example  2;  Let 

Q(x)  •  2Xj2  +  3x22 
•j'i  >  bj' 


J  *  1, . . . 5 


<8.  2) 
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where 


"11 


J2 


2 

X 

-1 

-2 

4 


4 

6 

-2 

-10 

-4 


The  polygon  defined  by  the  inequalities  lies  entirely  in  the  positive  quadrant,  so 
that  no  solution  of  the  form  (x,,  0)  or  (0,  Xg)  exists.  The  modified  problem  is 


Q(x)  -  x02+  2xx2+  3x22 


(6.3) 


X°  +  ajl*l  +  *j2x2  -  **j' 


(6.4) 


where  x1  *  (x  ,  x,,  x„). 

—  O  1  It 


For  the  first  stage,  upon  setting  xl  “  *2  *  °>  the  solution  xQ  *  max^  b^  1  bg  •  6, 
and  the  terminal  restriction  set  {2}  are  obtained. 

The  procedure  is  continued  just  as  in  Example  1  obtaining,  at  the  end  of  the  third 
stage,  the  solution  xQ  *  x,  *  4/3,  Xg  ■  2/3,  and  the  terminal  restriction  set  {1,  2}. 


Therefore,  to  drive  x„  to  zero,  take 
o 

Q(x.  t)  •  (t+1)  xQ2  +  2Xj2  +  3x22 


and  lQ  «  {1,  2} ,  and  obtain 


52  12  +  40t  ..  .  -4  +  30t 

xo  *  14  +"25t  '  X1  *  14  4  25t  '  x2  14  +  25t 


38t  -  12  ,  _  12  +  14t 

X1  *  14  +  25t  '  X2  14  +  2 St  • 


(6.5) 


Condition  (3. 4)  is  not  satisfied  by  X,  or  Xg  for  t  >  0,  while  from  (3.  5) 


14b1  -  12an  +  4a,  g  -  52 

*  '  -25b]  +  46ajJ  +  36^2 - 


i 
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which  is  negative  for  J  ■  3, 4,  5.  Thus,  the  necessary  condition  for  a  transition  is 
not  satisfied  for  t  >  0.  Letting  t  —  »  gives,  in  the  limit,  xQ  *  0,  Xj  *  8/5,  x2  * 

6/5,  where  the  latter  two  are  the  solution  to  the  original  problem  (6.  2). 

7.  DEGENERACY  2:  A  not  of  full  rank 

The  matrix  A  is  a  variable  matrix  depending  on  J(t);  i.  e. ,  A  haa  rows  a^', 
j  <  I(t).  A  Type  2  degeneracy  occurs  when,  for  some  particular  restriction  set  L, 

A  does  not  have  full  row  rank,  so  that  X(t,hlj)  »  (AR" 1  (t)A')  -1b  and  x(t,  hl^)  * 

Ft'^t)  A'X  fail  to  exist.  This  degeneracy  will  announce  itself  by  the  'blowing  up'  of 
the  computation  of  Xft.hlj)  and  xft.hlj);  i.  e. ,  at  some  point  in  the  computation  one 
will  be  attempting  to  divide  fay  zero. 

Assume  that  1  is  the  first  index  for  which  this  degeneracy  occurs  and  that  1  > 

2;  the  case  i  ■  1  is  treated  slightly  differently.  Assume  that  t  is  decreasing  and 

let  e  >  0  satisfy  tj  j  >  tt  +  e  if  t^j  >  tj  or  tj_2  >  *j  +  «  if  tjwl  *  tji  according  to  the 

definition  of  a  transition  point,  these  are  the  only  alternatives.  Both  cases  are 

treated  alike,  but  for  definiteness  assume  tj_j  >  tj.  t  is  fixed  at  the  value  tj  +  c,  bo 

that  I(ti  +  «)  ■  Il_1;  and  now.  since  i  is  the  first  index  for  which  Type  2  degeneracy 

occurs,  it  follows  that  A  has  full  row  rank.  Next  introduce  the  new  variable  x„  and 

o 

pew  parameter  s  as  in  the  first  degeneracy,  except  that  in  adding  xQ  to  the  restric¬ 
tions,  its  accompanying  coefficients  must  be  such  that  A  has  full  row  rank.  For 
this  purpose  it  is  desirable  to  have  available  rules  for  generating  linearly-independent 
columns,  as, for  example,  the  columns  of  the  identity  matrix. 

In  case  i  *  1  and  p  «  «,  i.e.,  the  first  subproblem  of  a  stage,  one  cannot  fix  t  at  tj+  c; 
it  is  necessary,  instead,  to  go  back  to  the  previous  subproblem  and  introduce  xQ  and 
s  before  t  reaches  zero.  This  is  the  only  way  the  case  i  *  1  may  differ  from  the  casei>l. 

If  t  is  increasing,  fix  t  at  t^  -  e,  «  >  0,  but  otherwise  the  procedure  is  the 
same. 

Example  3:  To  the  inequalities  of  Example  1  add  a  seventh,  5x^  +  7x2  >  17. 

In  the  first  aubproblem  of  the  second  stage,  when  t  *  t2  *  10,  5Xj  +  7x2  =  17,  since 
Xj  ■  2,  x2  »  1  at  t2.  In  this  case  I2  ■  {1,  2,7},  and,  having  3  elements,  degeneracy 
results.  Fixing  t  at  11  and  introduce  xQ  and  s,  the  new  problem  is 

Q(x0,x,  s)  =  <l+s)xo2  +  3xj2  +  (2+t)x22,  t*  11 

xo  +  V--bj'  . 7 

with  the  restriction  set  {1} .  The  solution  is 
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52ii+fL  x  ,  24<H-s>  ,  156(1^8) 

54  +  25s  '  x2  64  +  258  *  *1  64  +  25s  * 


(7.  1) 


The  largest  s  satisfying  either  (3,  4)  or  (3.  5)  is  168.  Therefore,  fixing  s  at 
179,  t  is  allowed  to  resume  the  role  of  parameter.  The  restriction  set  I  j  is  now 
{l},  and  the  successive  results  are  t2  *  10.  934,  I2  -{1,  7};  tg  »  6.  52,  Ig  «  {7>;  t4  » 

6.  208,  I4  •  {2,  7};  tj.  *  3.  124,  Ig  -  {2};  tg  -  1.  051,  Ig  -  (2.  3};  and  finally  t?  -  0,  Iy  • 
{2, 3}.  At  this  point  8  is  driven  back  to  -  (though  it  could  have  been  done  earlier)  by 
fixing  t  at  0  and  letting  s  be  the  parameter  with  range  179  <  a  <  «,  and  with  I  «{2,  3}. 
There  are  no  transitions  ass-*«,  and,  in  the  limit,  Xj  •  xg  «  3/2  is  obtained,  as 
in  Example  1. 


8.  DEGENERACY  3:  Condition  IV  of  Proposition  2  not  satisfied 

The  degeneracy  announces  itself  by  the  failure  of  the  candidate  J  to  satisfy  (iv) 
or  (v).  This  degeneracy  is  resolved  in  the  same  way  as  in  the  previous  degeneracy. 

9.  COMPUTATION  OF  x  (t,  h  and  *  (t,  h  I,) 

In  the  simple  examples  given  above  xft.hlj)  and  X(t,  hlj)  were  obtained  as  explicit 
function  of  t  by  the  straitfitforward  solution  of  equations.  While  this  is  easy  to 
do  for  a  few  variables,  it  becomes  very  time  consuming  for  larger  numbers  of  vari¬ 
ables  because  of  the  presence  of  the  parameter  t,  which  precludes  the  use  of  ordi¬ 
nary  numerical  methods.  Fortunately,  however  s>  relatively  simple  formula  for 
x(t,  hlj)  and  X(t,  hl^)  can  be  obtained. 

j.Rk^'  1  2.  given  by  (4.  1)  and  (4.  1.  1)  can  be  written: 

jRk(t)‘  *  2Rk(0)  +  tEj*  i  ’  2  (9.  1) 

where 


The  equations  for  x  -  xft.hlj)  and  \  ■  Mt.hlj)  are,  from  (2.  2)  and  (2.  3) 
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[R(t> 
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where  A,  which  implicitly  depends  on  I^,  is  always  assumed  to  be  of  full  row  rank. 
Letting  R(t)  •  it  follows,  from  (9.  1)  that 
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and,  hence, 


"R 

A’l 

i5l  J 

I  x  £ 

LA 

Oj 

X  1 

L'iJ 

(9.  4) 


where,  to  simplify  the  notation,  R  »  jRk(0)  and  E  *  E,.  Since  t  appears  linearly 
in,  at  most,  on  row  and  one  column  of  R(t),  it  follows  that  the  coordinates  of  x 
and  X  will  be  rational  in  t,  of  degree  two  at  most.  Therefore,  they  can  be  repre¬ 
sented  by: 


(9.5) 


where 


X  *  X2t2  +  t  +  , 

A  -  Ajt2  +  Ajt+  A^, 

y  *  y2‘2  +  +  y0  • 


(9.  6) 


Then  (9.  4}  becomes 
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By  hypothesis,  .  Q  is  nonsingular,  hence,  (9.  7)  is  equivalent  to: 
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(9.  7) 


(9.  8) 


Equations  (9.  7)  and  (9.  8)  are  used  to  determine  the  unknown  coefficients  in  (9.  6). 

As  a  solution  to  (9.  3)  exists  for  t  ■  0,  it  follows  that  yQ  j  0;  hence,  we  may 
set  yo  *  1  since  the  coefficients  in  (9.  6)  are  only  specified  up  to  a  multiplicative 
constant.  Since  (9.  7)  and  (9.  8)  hold  for  an  infinite  set  of  t  values,  the  coef¬ 
ficients  of  like  powers  of  t  on  either  side  of  these  equations  must  be  identical. 
Therefore,  from  (9.  8), 
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(9.  9) 


'*o  *  Z22* 

and,  from  j(9.  7), 


AX2*by2.  AXj-byj. 


(9.10) 
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The  remaining  equalities  from  (9.  7)  are  not  needed.  Solving  (9.  9)  in  terms  of  yj 
and  y2  yields 


So*Z12^ 


?1  *  ylZ12-  '  Z11EZ12- 

—2  3  y2Z12-  '  ylZllEZl2— +  Z11EZ11EZ12- 


*o  "  *Z22- 


(9.11) 


- 1  *  "y  1Z22^  +  Z21EZ1 2- 
—  2*  "y2Z22^  +  ylZ21EZl 2- "  Z21EZ11EZ12- 
2  *  E<y2zi22  -  yizuEZi22  +  znE2nEZ12y- 


The  last  equation  is  of  rank  0,  1,  or  2.  If  0,  set  y2  *  y^  *  0:  if  1  set  y2  ■  0  and 
solve  for  y^  (setting  yj  *  0  if  its  coefficient  is  0);  if  2  solve  for  y^  and  y2.  Making 
use  of  (9.  10),  a  solution  of  (9.  11)  is  obtained  in  each  case.  Since  any  solution  of 
(9.  11)  is  a  solution  of  (9.  8)  for  all  t,  and  (9.  8)  and  (9.  2)  are  equivalent  systems, 
it  follows  that  a  solution  for  (9.  2)  is  obtained.  This  solution  must  be  unique,  since 
the  matrix  on  the  left  side  of  (9.  2)  is  nonsingular. 

Each  time  there  is  a  transition  or  a  degeneracy 
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(9.12) 


changes,  and  these  changes  are  of  the  following  types. 

(1)  Add  or  subtract  one  or  more  rows  from  A,  corresponding  to  transitions 
in  which  Props.  1  or  2  are  satisfied. 

(2)  Border  R^,(0)  with  a  row  and  column  and  add  a  column  to  A  in  the  same 
position  as  the  column  added  to  RgjiO).  This  occurs  when  we  add  a  new 
variable  either  to  start  a  new  stage  or  to  resolve  a  degeneracy. 

(3)  The  reverse  of  (2)  occurs  when  we  delete  a  variable  previously  added 
to  resolve  a  degeneracy. 

(4)  Replace  the  zeros  in  the  kj'th  and  jk'th  positions  of  Rkj(0)  hy  rki  and 
rjg(»rkj),  respectively.  This  occurs  when  we  pass  from  one  subproblem 
to  the  next,  after  the  first  in  the  k'th  stage. 
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All  these  changes  can  be  accomplished  by  modifying  (9.  12)  by  well-known  methods 
rather  than  by  reinverting  each  time. 

10.  EXTENSION  TO  LINEAR  PROGRAMMING 

The  extension  is  obtained  by  embedding  the  linear  objective  function  in  a  second- 
degree  polynomial  with  a  positive  definite  quadratic  component,  solving  the  result¬ 
ing  problem  by  the  algorithm  just  given  and  then  driving  the  quadratic  part  to  zero 
in  the  final  stage.  The  algorithm  is  unchanged  but  some  changes  are  required  in 
the  solution  for  x  and  X. 

If  the  original  linear  programming  problem  is:  minimize  u'x  subject  to  a.  'x  > 

b. ,  i  =  1 . p,  where  u,  aj,  and  b^  are  all  given,  replace  this  problem  with  the 

derived  problem:  minimize  x'x  +  u'x  subject  to  the  same  conditions.  Condition 
(2.  2)  is  replaced  by  R(t)x  -  A'X  *  u;  (9.  4),  (9.  7)  and  (9.  8)  are  replaced  by  -tEx  by 
u  -  tEx;  and  in  (9.  9)  Xj  *  Z12b  and  -  *  Z22l>  are  replaced  by,  respectively, 

s  ZjjU  +  ZJ2b  and  =  Zj^u  +  Z22b.  The  required  changes  in  (9.  11)  are  then 
easily  made. 

After  the  solution  to  this  problem  has  been  found,  drive  x'x  to  zero  by  replacing 
x'x  +  u'x  by  x'x  +  tu'x  and  letting  t  go  from  1  to  •».  (9.  4)  is  now  replaced  by 
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where  E  ■  0.  Then,  simply, 
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If  there  are  n  variables  then  there  are  at  least  n  +  1  subproblems,  but,  at 
most,  only  a  few  more  than  n  +  1:  n  for  the  solution  of  the  derived  problem,  1 
tor  the  elimination  of  the  quadratic  component,  and  possibly  a  few  more  due  to 
degeneracies. 
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